Controlled rate vod server

ABSTRACT

A distributed stream processing system including a plurality of storage devices, at least one of which is configured to process at least one segment of a data stream, thereby producing a changed segment of the data stream, and provide output of at least the changed segment of the data stream, and a server configured to accept the output of the storage devices and combine the output into a transport stream. A method for combining a plurality of data streams including using one or more storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream, and providing output of at least the changed segment of the data stream, and combining the output of the storage devices into a combined data stream. Related apparatus and methods are also described.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/933,435 filed on Sep. 20, 2010, which is National Phase of PCT PatentApplication No. PCT/IL2008/000377 having International filing date ofMar. 18, 2008. The contents of the above Application is incorporatedherein by reference.

FIELD AND BACKGROUND OF THE INVENTION

The present invention, in some embodiments thereof, relates to a digitalvideo server and, more particularly, but not exclusively, to a digitalvideo server for video-on-demand services.

Digital video is often distributed as streams of digital data. One wellknown protocol for video data is MPEG.

Video streams, such as MPEG streams, typically have a Variable Bit Rate(VBR). The video streams are typically compressed, and the amount ofcompression depends on complexity of the scene, and on the amount ofchanges between scenes.

The MPEG video compression protocols compress video by reducing pixeldata redundancy within an image, and also redundancy between neighboringimages. MPEG video compression groups several consecutive images into aGroup Of Pictures (GOP), and defines most of the pictures, also termedvideo frames, by their difference from a neighboring picture, which isalso in the GOP.

A GOP contains X amount of data for N video frames, which need to beplayed out at a rate of R frames per second, thereby having a bit rateproportional to X*R/N. If there is need to lower the bit rate of a videostream, pictures in a GOP need to be changed together, since thepictures depend on each other for reconstruction.

In cases where video streams are distributed to customers, several videostreams are sometimes combined in one transport stream, termed aMultiple Program Transport Stream (MPTS). In some cases a transportstream contains only one program. Such transport streams are termedSingle Program Transport Streams (SPTS).

One example of video distribution, which is becoming more and morecommon nowadays, is distribution of Video On Demand (VOD). Customersrequest to receive specific videos, and VOD suppliers transmit eachvideo requested by the customers. For example, the VOD suppliertransmits several MPEG streams, some of different videos, and some ofthe same videos started at different times.

A transport stream often has a Constant Bit Rate (CBR). The bit rate ofthe transport stream is often a physical upper limit of some device inthe transmission path. In other cases the transport stream has a bitrate assigned to it by a service provider which performs thetransmission. A VOD supplier may purchase one or more units of such andsuch bandwidth, to use for supplying video on demand.

Combining VBR streams into a CBR transport stream can be done. Care mustbe taken that the total bit rate of the combination of VBR streams neverexceeds the CBR limit, otherwise the CBR transport stream may wellcrash, or be cut off by the service provider which performs thetransmission. Sometimes, the combining combines the streams by takingeach of the constituent VBR streams into account as if it constantly hasa maximum bit rate, and whenever the total bit rate of the combined VBRstreams falls below the CBR of the transport stream, padding is added.

Lowering a bit rate of a video stream can be done. It is a computationintensive task. MPEG streams, for example, are compressed for ease ofdecompression, having a relatively computationally simple decompression,and a relatively computation intensive compression. Therefore bit ratelowering of MPEG streams, which involves decompression (easy) andsubsequent compression (hard) is computationally intensive.

SUMMARY OF THE INVENTION

An aspect of some embodiments of the invention relates to using adistributed data storage system for distributed processing of datastored on the distributed storage nodes. Some types of processing arecomputation intensive, such as, by way of a non-limiting example, videostream processing. One or more computers may control the processing,instructing data storage nodes, which also have some processingcapability, to perform some or all of the processing.

An aspect of some embodiments of the invention relates to controlling abit rate of a transport stream by controlling the bit rate of componentsof the transport stream. Changing bit rate can be computation intensive,and changing the bit rate of components of the transport stream isperformed in a distributed fashion, so as to spread the computation loadaway from just one computer.

The changing of bit rate may be done in real time, that is, just priorto transmitting the transport stream, or may be done ahead of real time,and changed bit rate stream components can be saved until needed.

According to an aspect of some embodiments of the present inventionthere is provided a distributed stream processing system including aplurality of storage devices, at least one of which is configured toprocess at least one segment of a data stream, thereby producing achanged segment of the data stream, and provide output of at least thechanged segment of the data stream, and a server configured to acceptthe output of the storage devices and combine the output into atransport stream.

According to some embodiments of the invention, the process includeschanging the bit rate.

According to some embodiments of the invention, the at least one storagedevice is further configured to store at least the changed segment ofthe data stream.

According to some embodiments of the invention, the server is configuredto provide bit rate control instructions to the plurality of storagedevices, and the plurality of storage devices are configured to acceptthe bit rate control instructions from the server and to change the bitrate of the segment of a data stream according to the instructions.

According to some embodiments of the invention, at least one of the datastreams is a video stream. According to some embodiments of theinvention, the process includes transcoding the video stream. Accordingto some embodiments of the invention, the process includes insertingcontent into the video stream. According to some embodiments of theinvention, the process includes producing video effects in the videostream. According to some embodiments of the invention, the processincludes content protection processing with respect to the video stream.According to some embodiments of the invention, the process includessearching in the video stream.

According to some embodiments of the invention, the storage devices usea data protection scheme to store the changed segment of the datastream.

According to some embodiments of the invention, further includingproducing and storing metadata including information about a bit rate ofthe changed segment of the data stream.

According to an aspect of some embodiments of the present inventionthere is provided a method for combining a plurality of data streamsincluding using one or more storage devices for processing at least onesegment of a data stream, thereby producing a changed segment of thedata stream, and providing output of at least the changed segment of thedata stream, and combining the output of the storage devices into acombined data stream.

According to some embodiments of the invention, the processing includeschanging a bit rate.

According to some embodiments of the invention, further includingstoring the at least the changed segment of the data stream.

According to some embodiments of the invention, further includingproviding bit rate control instructions to the plurality of storagedevices, and the storage devices accepting the bit rate controlinstructions from the server and changing the bit rate of the segment ofa data stream according to the instructions.

According to some embodiments of the invention, at least one of the datastreams is a video data stream. According to some embodiments of theinvention, the processing includes video stream processing.

According to some embodiments of the invention, further including, afterthe changing, storing the changed segment of the data stream, andperforming the providing upon a request from the server.

According to some embodiments of the invention, further includingproducing and storing metadata including information about a bit rate ofthe changed segment of the data stream.

According to an aspect of some embodiments of the present inventionthere is provided a system configured to receive a transport streamincluding a plurality of data streams including a server configured tosplit the transport stream into a plurality of data streams, a pluralityof storage devices configured to receive and store one or more of thedata streams from the server, at least one of the storage devices beingfurther configured to process at least one segment of a data stream,thereby producing a changed segment of the data stream, and store atleast the changed segment of the data stream.

According to some embodiments of the invention, the process includeschanging the bit rate.

According to some embodiments of the invention, at least one of the datastreams includes a video stream, and the process includes video streamprocessing.

According to an aspect of some embodiments of the present inventionthere is provided a method for receiving a transport stream including aplurality of data streams including splitting the transport stream intoa plurality of data streams, storing the data streams on a plurality ofstorage devices, using at least one of the storage devices forprocessing at least one segment of a data stream, thereby producing achanged segment of the data stream, and storing the changed segment ofthe data stream.

According to some embodiments of the invention, the processing includeschanging a bit rate.

According to some embodiments of the invention, at least one of the datastreams is a video stream, and the processing includes video streamprocessing.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

Implementation of the method and/or system of embodiments of theinvention can involve performing or completing selected tasks manually,automatically, or a combination thereof. Moreover, according to actualinstrumentation and equipment of embodiments of the method and/or systemof the invention, several selected tasks could be implemented byhardware, by software or by firmware or by a combination thereof usingan operating system.

For example, hardware for performing selected tasks according toembodiments of the invention could be implemented as a chip or acircuit. As software, selected tasks according to embodiments of theinvention could be implemented as a plurality of software instructionsbeing executed by a computer using any suitable operating system. In anexemplary embodiment of the invention, one or more tasks according toexemplary embodiments of method and/or system as described herein areperformed by a data processor, such as a computing platform forexecuting a plurality of instructions. Optionally, the data processorincludes a volatile memory for storing instructions and/or data and/or anon-volatile storage, for example, a magnetic hard-disk and/or removablemedia, for storing instructions and/or data. Optionally, a networkconnection is provided as well. A display and/or a user input devicesuch as a keyboard or mouse are optionally provided as well.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIG. 1 is a simplified illustration of a system constructed andoperative according to an exemplary embodiment of the invention;

FIG. 2 is a simplified flow chart illustration of a method operativeaccording to an exemplary embodiment of the invention;

FIG. 3 is a more detailed illustration of an example embodiment of thesystem of FIG. 1;

FIG. 4 is a simplified graphical illustration of variable bit rates in avideo stream; and

FIG. 5 is a simplified graphical illustration of how chopping off bitrate peaks reduces a maximal bit rate of a video stream in an embodimentof the invention.

DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

The present invention, in some embodiments thereof, relates to a digitalvideo server and, more particularly, but not exclusively, to a digitalvideo server for video-on-demand services.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

An aspect of some embodiments of the invention relates to using adistributed data storage system for distributed processing of datastored on the distributed storage nodes. One or more computers maycontrol the processing, and data storage nodes, which also have someprocessing capability, perform at least some of the processing.

Some types of processing are computation intensive, such as, by way of anon-limiting example, video stream processing. Non-limiting examples ofcomputation intensive video stream processing include:

1. Rate shaping video streams, that is, changing the bit rate of videostreams. Often the changing is a lowering of the bit rate of a videostream in order to fir the video stream into a narrower bandwidth.

2. Transcoding video streams, from one format to another, such as, byway of a non-limiting example, MPEG2 to MPEG4 or vice versa, HighDefinition (HD) to Standard Definition (SD) or vice versa.

3. Content processing, such, by way of a non-limiting example, insertingadditional content such as a banner into a video stream, a logo into avideo stream, producing a Picture-In-Picture (PIP) video stream, andadding a voice overlay or a video overlay.

4. Video effect processing, such as, by way of a non-limiting example,fade-in, fade-out, alpha blending, changing brightness, and a wipeeffect.

5. Content protection processing, such as, by way of a non-limitingexample, encryption, decryption, inserting a watermark, deleting awatermark.

6. Searching within a video stream, such as, by way of a non-limitingexample, searching for a caption, and/or other searchable attributes.

Some embodiments contemplated include combining data streams stored ondata storage nodes into a transport stream, while processing at leastsome of the data streams.

Some embodiments contemplated include receiving a transport streamcontaining data streams, splitting the transport stream into datastreams for storage in storage devices, while having some of the storagedevices process at least part of the data streams, and storing theprocessed part.

Examples of specific embodiments are provided herein with reference toVariable Bit Rate (VBR) video streams, such as MPEG video streams, beingcombined into a Constant Bit Rate (CBR) transport stream. The examplesare to be understood in a broader context of VBR data streams beingcombined into a CBR transport stream.

Other specific embodiments contemplated include receiving a transportstream containing video streams, splitting the transport stream intovideo streams for storage in storage devices, and instructing some ofthe storage devices to process at least part of the video streams andstore the changed, processed part.

An aspect of some embodiments of the invention relates to controlling abit rate of a transport stream by controlling the bit rate of videostream components of the transport stream. Changing a bit rate of avideo stream can be computation intensive, and changing the bit rate ofthe video stream components is performed in a distributed fashion, so asto spread the computation load.

It is noted that often video streams are stored on distributed datadisks, and combined into a transport stream by a streamer, which is adedicated computer. The data disks and the streamer are likely to beexpensive pieces of equipment, dedicated to their intended tasks. Analternative approach to the above is using a lower cost general purposecomputer, also termed a server, as a streamer, and using low coststorage on low cost general purpose computers as the distributedstorage. The alternative approach produces an additional advantage,which is that the distributed storage also has processing power, openingthe door to distributed processing.

An aspect of some embodiments of the invention relates to other types ofprocessing of the transport stream, being performed in a distributedfashion. The invention is suitable for distributing processing of streamcomponents.

The term “changing of bit rate” in all its grammatical forms is usedthroughout the present specification and claims interchangeably with theterm “rate shaping” and the term “rate reducing” and their grammaticalforms.

The term “video stream” in all its grammatical forms is used throughoutthe present specification and claims interchangeably with the terms“stream”, “data stream”, “data file”, “file”, “MPEG stream”, and “MPEGvideo stream” and their grammatical forms.

It is noted that the term video stream can refer to a stream which islimited in time, such as, by way of a non-limiting example, a movie, atelevision program, and a 30 second commercial, and the term can referto a video stream which is not limited in time, such as a 24 hour a dayprogram.

Apparatus which combines streams into a transport stream is termedherein a streamer.

Reference is now made to FIG. 1, which is a simplified illustration of asystem constructed and operative according to an exemplary embodiment ofthe invention.

The system comprises a streamer 100, connected to one or more storageclusters 105.

The streamer 100 is configured to provide an output 120 or outputs 120of one or more transport streams.

The streamer 100 requests the storage clusters 105 to send it videostreams (not shown), and combines the video streams into the one or moretransport streams.

If there is a need for processing to be performed on one or more of thevideo streams, the streamer 100 instructs one or more of the storageclusters 105 to perform the processing and provide a changed, processed,video stream for the combining By way of a non-limiting example, thestreamer 100 tracks the bit rate of each of the transport streams. If abit rate of a transport stream is about to exceed a pre-defined limit,the streamer 100 ensures, as will be further described below, that thebit rate of one or more of the video streams combined into the transportstream is lowered, thereby ensuring that the bit rate of the transportstream does not exceed the limit.

The streamer 100 instructs one or more of the storage clusters 105 toprovide a version of one or more of video streams having a lower thannominal bit rate.

The storage clusters 105 provide the streamer 100 with a lower bit rateversion of the one or more video stream which the streamer instructed tobe at a lower bit rate.

Reference is now made to FIG. 2, which is a simplified flow chartillustration of a method operative according to an exemplary embodimentof the invention.

FIG. 2 depicts a flow chart of the method used in the system of FIG. 1,which uses one or more storage devices (150) for processing at least onesegment of one of the data streams (160). An optional method ofprocessing used by an embodiment of the invention is described furtherbelow with reference to FIG. 5.

Output of at least the changed data stream is provided (165), asdescribed above with reference to FIG. 1, and will be described furtherbelow with reference to FIG. 3.

A streamer is used for combining the output of the storage devices intoa combined data stream (170), as described above with reference to FIG.1, and will be described further below with reference to FIG. 3.

Reference is now made to FIG. 3, which is a more detailed illustrationof an example embodiment of the system of FIG. 1.

The system comprises a streamer 100, connected to one or more storageclusters 105. The storage clusters 105 each contain one or more storagemachines 110, each of which is connected to one or more storage devices115.

The streamer 100 optionally instructs one or more of the storageclusters 105 to provide a version of one or more of video streams havinga lower than nominal bit rate.

The storage clusters 105 optionally provide the streamer 100 with alower bit rate version of the one or more video streams which thestreamer instructed to be at a lower bit rate.

The Streamer 100

The streamer 100 combines several video streams into a transport stream.The streamer optionally ensures that a sum of the bit rates of the videostreams does not exceed a bit rate limit associated with the transportstream. Combining VBR video streams into a CBR transport stream, whenthe total bit rate of the video streams is less than the bit rate limit,is known in the art. For example, one way of doing so, is tracking thecombined bit rate, and adding null data to the transport stream asneeded to bring the bit rate of the combined transport stream up to aconstant value.

A more efficient use of a CBR transport stream is to combine as manyvideo streams as possible into the transport stream. Since the videostreams show a variable bit rate, it is possible to combine the videostreams such that their combined bit rate will usually be below thelimit of the transport stream. When the combined bit rate of the videostreams is expected to exceed the limit, the streamer 100 optionallyreplaces some or all of the video streams with lower bit rate versionsof the video streams, and prevents the exceeding.

The average bit rate of each VBR video file or video file segmentparticipating in a transport stream is optionally derived based on thefollowing parameters: a modulation type for the MPTS, such as, by way ofa non-limiting example, Quadrature Amplitude Modulation (QAM) type suchas 64/128/256 Symbol Rate; and the number of streams participating inthe MPTS.

Reference is now made to FIG. 4, which is a simplified graphicalillustration of variable bit rates in a video stream.

FIG. 4 depicts a graph 200 of bit rate of a segment of a video streamvs. time. The graph is of a segment of an original video stream with anoriginal, unmodified, bit rate. The horizontal axis 205 of the graph 200depicts time, in units of video frames. Since video is typicallydisplayed at a constant frame rate, frames are directly proportional totime. The vertical axis 210 of the graph 200 depicts bit rate, in unitsof Mbps.

The segment contains several GOPs 215, 216, 217, 218. Each of the GOPs215, 216, 217, 218, comprises individual pictures, one bar of the graph200 standing for each picture. Each of the pictures has an inherent bitrate associated, based on the amount of data required to portray thepicture, and a frame rate of the video stream. The inherent bit rate isdepicted by the relative height of the bar which stands for the picture.

Since each of the GOPs 215, 216, 217, 218 is compressed together, eachGOP is also associated with a bit rate. The bit rate of a GOP is basedon the amount of data, after compression, which the GOP contains, aframe rate of the video stream, and the number of pictures within theGOP.

GOP bit rates 220, 221, 222, 223 are depicted by dotted linescorresponding to the bit rates of the GOPs 215, 216, 217, 218respectively. The GOP bit rates 220, 221, 222, 223 are not equal,depending as they do on contents of the pictures of the video stream.

A maximal bit rate of the segment in the illustration of FIG. 4 is theGOP bit rate 221, associated with the GOP 2 216.

Reference is now made to FIG. 5, which is a simplified graphicalillustration of how chopping off bit rate peaks reduces a maximal bitrate of a video stream in an embodiment of the invention.

FIG. 5 depicts the segment of FIG. 4, in which bit rate peaks ofindividual pictures have been modified by “chopping off”, that isreducing, just the bit rate peaks. The values of the unmodified picturebit rates and the unmodified GOP bit rates 220, 221, 222, 223 are stilldepicted in FIG. 5, for purpose of comparison and to aid understanding.

A picture in GOP 2 216, which had a highest inherent bit rate of thepictures of GOP 2, was reduced in bit rate by a first amount 225. A GOPbit rate 231 of GOP 2 216 is lower than the GOP bit rate 221 of GOP 2216 before bit rate modification.

A picture in GOP 3 217, which had a highest inherent bit rate of thepictures of GOP 3 217, was reduced in bit rate by a second amount 226,different from the first amount 225. A GOP bit rate 232 of GOP 3 217 islower than the GOP bit rate 222 of GOP 3 217 before bit ratemodification.

Two pictures in GOP 4 218, which had the two highest inherent bit rateof the pictures of GOP 4 218, were reduced in bit rate by a third amount227 and a fourth amount 228. A GOP bit rate 233 of GOP 4 218 is lowerthan the GOP bit rate 223 of GOP 4 218 before bit rate modification.

The files which together make up a transport stream change over time.For example, when the streamer is a VOD server, sessions are allocatedby end users, and the streamer 100 is required to read files from thestorage cluster 105. The streamer instructs each processing unit 110 inthe storage cluster 105 to reduce the rate of certain GOPs according toa statistical multiplexing algorithm.

Since rate shaping can be computationally intensive, the streamer 100optionally causes rate shaping of video streams to be performed bycomputing power available in the storage clusters 105 (FIG. 1). Whilestorage clusters 105 typically contain a large amount of storage, whichis needed for storing the video streams, the storage clusters arenevertheless capable of computation. Having several storage clusters 105enables using the computation power built into a distributed videostorage system for also rate shaping the video.

It is noted that, similarly to the example of FIG. 5, rate shaping isoptionally not performed on an entire file, but rather on a filesegment. The size of the file segment ranges from an entire file, to aportion of a file, down to a GOP.

Statistical Multiplexing (Statmux) Decision Making

Upon having an MPTS or a group of SPTS streams directed at a specificoutput with a specified maximum allowed bit rate, the streamer 100(FIG. 1) starts reading one or more streams from one of the storageclusters 105.

Along with the streams being read, the streamer 100 reads metadataassociated with each stream. More detail about the metadata is providedbelow, with reference to “Ingesting a Video File”.

Based at least partly on the metadata, the streamer 100 can predict whatwill be the aggregated bit rate of all streams in the MPTS, or group ofSPTS streams going into a specific output. The metadata also containsinformation about complexity of each stream, and enables the streamer100 to predict a level of deterioration in video stream quality due torate reduction.

By taking the metadata parameters into consideration, the streamercompiles a set of commands to the storage clusters 105, requesting thestorage clusters 105 for relevant data segments with a specific bit rateper GOP.

In some embodiments of the invention, the streamer 100 is a componentcombining the streams into the data stream, and the component whichconsiders the metadata and issues commands to the storage clusters 105is a stream management server (not shown). The streamer 100 and thestream management server may or may not be separately housed and/orseparately located.

SPTS

For legacy transport streams which do not support MPTS as their input,the statistical multiplexing method described above may be distributedamong several SPTS and several streamers 100. Each streamer 100 managesone or more transport streams, and instructs relevant storage machines110, via one or more storage clusters 105, to reduce the rate of videostreams when necessary.

Each VBR program stream is padded with null stuffing into a CBR SPTS atthe maximum allowed bit rate, and sent to a suitable output.

A distributed statmuxing method optionally manages an aggregatedbandwidth of all streams going into a specific output transport stream,even when some video streams originate from different streamers machine.

Trick Play

During play out of a video stream, an end user may choose to use trickplay modes, such as Fast Forward (FF), rewind, and pause. This sectiondescribes performance of the invention in case at least one of the videostreams is played out using a trick mode.

Optionally, In FF and rewind modes, only a first I-frame from a GOP isrequested and received by the streamer 100, and not all pictures of theGOP. Managing the bit rate is optionally done by the streamer 100, byadding a suitable number of P-frames to I-frames and forming newlygenerated GOPs.

If the bit rate of a GOP comprising I-frames only is still too high, bitrate reduction is optionally requested by the streamer 100 from thestorage machines, in order to reduce the bit rate of the I-frame.Alternatively, the streamer 100 performs rate reduction for theI-frames.

In pause mode, the streamer 100 stops transmitting the video stream, sothere is no need to use rate reduction for a paused video stream.

The streamer 100 takes pause mode into consideration, and for example,in case of a paused video stream, other streams in the transport streammay use more bit rate.

Rate Shaping

Optionally, rate shaping is done in real time, or approximately in realtime, that is, just prior to transmitting the transport stream.Alternatively, the rate shaping is done ahead of real time, and changedbit rate video streams are saved until needed.

Whether the rate shaping is performed in real time or ahead of time,each of the storage machines 110 which performs rate shaping optionallyreceives instructions for rate shaping from the streamer 100, whichinclude a bit rate parameter which specifies to a maximum bit rate, orby how much to reduce the bit rate of a file segment. Alternatively,each storage machine 110 is configured for one or more configurablelevels of rate shaping.

Optionally, rate reduction reduces quality of images in frames of eachGOP, so that a maximum bit rate is not exceeded. Alternatively the ratereduction reduces the bit rate by a percentage of the bit rate, or bysubtracting a bit rate from an unmodified bit rate.

Reducing the bit rate is optionally performed over a period of time,corresponding to a certain segment of the video stream, and the amountof reduction may change after the period of time.

In some embodiments of the invention, each storage cluster 105 handlesan internal reallocation of a bit rate budget within each GOP, forexample, allocating more or less bit rate to different frames within aGOP based on the frame's complexity, optionally providing some areas ofthe frames with a specific treatment.

Rate Shaping in Approximately Real Time

When rate shaping is performed in real time or approximately real time,the streamer combines several video streams into the transport streambased, at least in part, on a total bit rate of the video streams andoptionally on a bit rate limit for the transport stream.

It is noted that transport streams sometimes carry more than just videostreams. The data carried by the transport stream beyond the videostreams is hereby considered a data stream, and the description of datastreams is taken to be included in the video stream examples.

Optionally, the streamer 100 estimates the bit rate of the transportstream, and optionally estimates the bit rate of each of the videostreams. Optionally, the estimation is done by the streamer actuallymeasuring the bit rate of the transport stream and optionally measuringthe bit rate of each of the video streams.

The streamer 100 instructs the storage machines 110 which video streamsto rate shape, and how to rate shape the video streams.

In some embodiments of the invention the streamer 100 instructs thestorage machines 110 which video stream and at what bit rate to providethe streamer. In other embodiments of the invention the streamer 100instructs the storage machines 110 which video stream to provide thestreamer 100, and by what amount, or by what percentage, to lower thebit rate of the video stream.

Optionally the storage machines 110 which rate shape a video stream,also cache the rate shaped video stream, so that of the video streamneeds to be output again as a rate shaped video stream, the cached rateshaped video stream is recovered from the cache.

Rate Shaping Ahead of Real Time

When rate shaping is done ahead of real time, the streamer 100optionally instructs the storage clusters 105 which video stream to rateshape, to what bit rate to shape the video stream, and where to storethe rate shaped video stream.

In other embodiments of the invention the streamer 100 instructs thestorage clusters 105 which video stream to rate shape, by what amount,or by what percentage, to lower the bit rate of the video stream, andwhere to store the rate shaped video stream.

When rate shaping is done ahead of real time, metadata is optionallyproduced, associating some or all of the saved video streams with anestimated or measured bit rate. Optionally the metadata includes the bitrate of segments of the video streams, rather than a bit rate for anentire video stream.

Optionally, the metadata includes the bit rate for each individual GOP.

Optionally, the metadata for a video stream or for a segment of a videostream includes a maximum bit rate as well as an average bit rate.

Optionally, a video stream is stored in more than one version, withdifferent versions having been rate shaped and comprising different bitrates. Optionally, only some segments of the video stream are rateshaped, the rate shaped segments preferably being the higher bit ratesegments.

Optionally, the segments of the video stream which are rate shaped areas small as one GOP.

Since in the example described above rate shaping is performed ahead ofreal time, there may not be certainty that rate shaped versions orsegments of a video stream will actually be used. In some embodiments ofthe invention deciding which video streams or segments thereof will berate shaped is performed according to projected statistics of videodemand. In some embodiments of the invention the deciding is performedaccording to actual video demand.

Various schemes for saving computation are contemplated, such as: rateshaping only video streams which are most in demand; rate shaping onlyhigh bit rate segments of video streams; rate shaping most-in-demandvideo streams in advance of real time, and rate shaping other videostreams in real time or approximate real time; and rate shaping a videostream in real time when first combined into a transport stream, andsaving the rate shaped video stream or rate shaped segments of the videostream, and their associated metadata, for later re-use.

Various schemes for saving storage space are contemplated, such as:saving rate shaped versions only of video streams which are most indemand; saving rate shaped versions of only high bit rate segments ofvideo streams; and saving rate shaped versions of a video stream onlyafter a first combining into a transport stream.

Storage

An exemplary embodiment of the invention includes a storage cluster 105,which comprises several storage machines, also termed data servers. Eachstorage machine has one or more storage devices, and some processingpower.

Optionally, the storage cluster 105 protects data by storing redundantdata. The protection is performed using schemes such a RAID. It is to benoted that the protection is configured per file segment. The filesegment corresponds to a video stream, segment of a video stream, andeven a GOP of a video stream.

Rate Reduction with Data Protection

In case there is data loss on a first one of the storage clusters 105,the streamer 100 requests rate reduced file segments from a second oneof the storage clusters 105. The second storage cluster 105 gathers therequired file segments from the second storage cluster 105. The secondstorage cluster 105 performs rate reduction on the recovered filesegments.

Fallback Copy

Optionally, each file is stored in a predefined rate reduced version onthe storage cluster. The version is optionally either a CBR version, ora capped VBR with a maximum bit rate which is not higher then theaverage bit rate of the original stream. The predefined rate reducedversion is termed a fallback copy.

In case there is data loss of an original version, or if there is noavailable processing power to reduce the rate of the original version,the fallback copy is optionally used. The fallback copy is alsooptionally used if the bit rate of the fallback copy falls inside thelimits of the bit rate required by the streamer 100.

Caching of Processed GOPs

Each storage machine 110 optionally has a cache containing already ratereduced GOPs. The cache is optionally based upon an LRU method. Someembodiments of the invention use a cached version of a rate reduced GOPrather than utilize rate reduction. The preference for using the cache,and a size of the cache, are determined based at least partly on theavailable processing power of the storage machine 110.

By way of a non-limiting example, when the streamer 100 requests acertain GOP at a bit rate of 2.5 Mbps, and if there is a cached versionof the same GOP reduced to 2.4 Mbps, the system optionally chooses touse the cached 2.4 Mbps version. This frees the storage machine 110 torate shape other GOPs that are not already cached.

The storage machine 110 optionally has a dedicated amount of storage andstorage IO for caching purposes.

Rate Reduction and “Flash-Crowd”

A data server may optionally get multiple requests for rate shaping afile segment, from multiple streamers, or optionally multiple requestsfrom a single streamer but intended to be combined in an MPTS the singlestreamer processes.

Such a scenario can be common when many users would like to watch aspecific file at substantially the same moment, for instance a newlyissued popular event. The data server which stores the requested videosegment receives many requests for the same segment, more than it canprocess. The data node reaches a point where it is not able to serve allthe requests and maintain an approximately real-time response.

In some embodiments of the invention, the data server aggregatesrequests and chooses to process only several rate shaping commands, suchthat will not exceed its processing power and its read-from-storagecapabilities.

In case a specific rate reduction request wasn't selected for real-timeprocessing, the data server optionally provides a copy of a closest rateshaped version, or a closest rate shaped version with less than therequired bit rate, available of the requested rate shaped segment.

Ingesting a Video File

An ingested video file is typically encoded to a VBR format. Thestreamer 100 parses the video file, and breaks the video file into filesegments.

In some embodiments of the invention the streamer 100 optionallyproduces and stores, optionally on one or more storage clusters 105,metadata about the stream behavior and complexity. The metadata isoptionally used later for statistical multiplexing decision making.

During the ingestion process, and optionally during real time rateshaping, the metadata for the video file is generated, and kept in aseparate file. The information saved in the metadata file containsinformation about the stream bit rate, frame structure and complexity.

Data Segments

Video streams are optionally striped across storage devices 115 in thestorage machines 110 and optionally also across storage machines 110 ina storage cluster 105.

Each file is optionally cut into segments which are optionallydistributed among the storage devices. Each segment contains one or moreGOPs belonging to the file.

It is to be noted that each storage device may have a segment from eachfile, and vice versa, segments from each file may appear on each storagedevice.

The striping and distribution of the files among the storage machines,coupled with the rate shaping requirements made of the storage machines,cause a distribution of the computation load among the storage machines.

The number of GOPs per segment is determined in order to optimizethroughput of a clustered file system. The number of GOPs per segment isdetermined based, at least partly, on storage parameters such as blocksizes in the storage device, on computation parameters such ascomputation speed on the storage machine, on the number of computationmachines, and on a bit rate of the segment. For example, when a segmentsize is determined to be 1 MB, the segment can contain 10 GOPs of aStandard Definition (SD) video stream, or 5 GOPs of a High Definition(HD) stream.

It is noted that when an optimal size of a segment is not completelyfilled by GOPs, padding is optionally applied.

In some embodiments of the invention, a segment is smaller than an MPEGGOP. A segment can be, by way of a non-limiting example, a frame; anumber of frames, with no relation to the underlying MPEG GOP; and evenpart of a frame, even as small as a slice of an MPEG macroblock.

It is noted that storing am MPEG GOP together can save time inassembling data for rate shaping.

It is noted that once a segment has been rate shaped, the rate shapedsegment can optionally be stored on one data server, and can optionallybe stored on more than one data server.

In some embodiments of the invention, especially where MPEG2 stream areconcerned, a segment is minimally chosen to be one or more GOPs, that isall frames from one I-frame to a next I-frame, which fit into and arealigned to a 1 MB data block.

Data Protection

Each segment of data written into storage is optionally protected by adata protection scheme, by way of a non-limiting example a dataprotection scheme using data redundancy, such as RAID. The level of RAIDprotection is any level of RAID protection, such as RAID level 0, 1, 2,3, 4, 5, and so on, and may optionally be different per file. Eachsegment is thus protected by redundant data, which is written into thestorage cluster, to different storage devices, on different storagemachines.

In some embodiments of the invention protection management is providedby the storage cluster. In some embodiments of the invention protectionmanagement is provided by each of the storage machines. In someembodiments of the invention protection management is provided by thestreamer.

A non-limiting example of protection functionality is now provided. Astreamer attempts to fetch a file segment from a first storage device ona first storage machine. Due to failure of the first storage device orthe first storage machine, the data segment is unavailable.

Recovery option 1: when the streamer manages the protection scheme, thestreamer fetches redundant data written by the protection algorithm on asecond storage device, and reconstructs the unavailable data. Thestreamer does not necessarily rate shape the reconstructed data. If rateshaping is necessary, the streamer optionally instructs the storageclusters providing other streams to provide the other streams at lowerbit rate.

Recovery option 2: when each of the storage machines manages theprotection scheme, the streamer requests a second storage machine tofetch the unavailable data. The second storage machine retrieves theredundant data, and reconstructs the previously unavailable data usingthe protection scheme. The storage machine then performs rate shaping onthe reconstructed segment, and deliver the rate shaped and recovereddata to the streamer.

Recovery option 3: when the streamer manages the protection scheme, thestreamer fetches redundant rate shaped data from the storage cluster,and reconstructs the unavailable data according to the protectionscheme.

Additional Features

The storage machine 110 optionally overlays text, video, and/or soundover file segments, according to instructions received from the streamer100. The overlay is optionally performed in the compressed domain.

Each storage machine 110 can optionally convert video segments from onevideo format to another, using a transcoding algorithm. The transcodingis optionally per data segment, so that each storage machine 110 is onlyrequired to transcode segments which are stored thereon.

It is expected that during the life of a patent maturing from thisapplication many relevant rate shaping methods will be developed and thescope of the term rate shaping is intended to include all such newtechnologies a priori.

The terms “comprises”, “comprising”, “includes”, “including”, “having”,and their conjugates, mean “including but not limited to”.

The term “consisting of” means “including and limited to”.

The term “consisting essentially of” means that the composition, methodor structure may include additional ingredients, steps and/or parts, butonly if the additional ingredients, steps and/or parts do not materiallyalter the basic and novel characteristics of the claimed composition,method or structure.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a video stream” or “at least one video stream” may include aplurality of video streams, including combinations thereof.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable sub-combination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

All publications, patents and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present invention. To the to extentthat section headings are used, they should not be construed asnecessarily limiting.

What is claimed is:
 1. A distributed stream processing systemcomprising: a plurality of storage devices, at least one of which isconfigured to: process at least one segment of a data stream, therebyproducing a changed segment of the data stream; and provide output of atleast said changed segment of the data stream, and a server configuredto accept the output of the storage devices and combine the output intoa transport stream.
 2. The system of claim 1 in which the at least onestorage device is further configured to store at least the changedsegment of the data stream.
 3. The system of claim 1 in which the serveris configured to provide bit rate control instructions to the pluralityof storage devices, and the plurality of storage devices are configuredto accept the bit rate control instructions from the server and tochange the bit rate of said segment of a data stream according to theinstructions.
 4. The system of claim 1 in which at least one of the datastreams is a video stream.
 5. The system of claim 4 in which the processcomprises transcoding the video stream.
 6. The system of claim 4 inwhich the process comprises inserting content into the video stream. 7.The system of claim 4 in which the process comprises producing videoeffects in the video stream.
 8. The system of claim 4 in which theprocess comprises content protection processing with respect to thevideo stream.
 9. The system of claim 4 in which the process comprisessearching in the video stream.
 10. The system of any one of claim 1 inwhich the storage devices use a data protection scheme to store saidchanged segment of the data stream.
 11. The system of claim 1 andfurther comprising producing and storing metadata comprising informationabout a bit rate of said changed segment of the data stream.
 12. Amethod for combining a plurality of data streams comprising: using oneor more storage devices for: processing at least one segment of a datastream, thereby producing a changed segment of the data stream; andproviding output of at least said changed segment of the data stream,and combining the output of the storage devices into a combined datastream.
 13. The method of claim 12 and further comprising storing the atleast the changed segment of the data stream.
 14. The method of claim 12and further comprising providing bit rate control instructions to theplurality of storage devices, and the storage devices accepting the bitrate control instructions from the server and changing the bit rate ofsaid segment of a data stream according to the instructions.
 15. Themethod of claim 12 in which at least one of the data streams is a videodata stream.
 16. The method of claim 12 in which the processingcomprises video stream processing.
 17. The method of claim 12 andfurther comprising, after the changing, storing said changed segment ofthe data stream, and performing the providing upon a request from theserver.
 18. The method of claim 12 and further comprising producing andstoring metadata comprising information about a bit rate of said changedsegment of the data stream.
 19. A system configured to receive atransport stream comprising a plurality of data streams comprising: aserver configured to split the transport stream into a plurality of datastreams; a plurality of storage devices configured to receive and storeone or more of the data streams from the server, at least one of thestorage devices being further configured to: process at least onesegment of a data stream, thereby producing a changed segment of thedata stream; and store at least the changed segment of the data stream.20. The system of claim 19 in which the process comprises changing thebit rate.